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The simple permutations in two permutation classes — the 321-avoiding 
permutations and the skew-merged permutations — are enumerated us- 
ing a uniform method. In both cases, these enumerations were known 
implicitly, by working backwards from the enumeration of the class, but 
the simple permutations had not been enumerated explicitly. In partic- 
ular, the enumeration of the simple skew-merged permutations leads to 
the first truly structural enumeration of this class as a whole. The exten- 
sion of this method to a wider collection of classes namely grid classes of 
infinite paths is discussed. 



1. Introduction 

Given permutations n and c, thought of as sequences of positive integers, we say that re contains 
o~, and write a < re, if n has a subsequence n{i\) ■ ■ ■ n(i^) of the same length as o~ which is order 
isomorphic to o~ (i.e. Tt(i s ) < 7t{it) if and only if o~(s) < c(f)); otherwise, we say that n avoids o~. 
Containment is a partial order on permutations, and is the only such order considered in this paper. 
For example, n = 391867452 contains o~ = 51342, as can be seen by considering the subsequence 
7r(2)7r(3)7r(5)7r(6)7r(9) = 91672. 

Given a collection X of permutations, we denote by X n the set of permutations in X of length n. 
The generating function of X is then 

= I ^ n I ^* ' 

neX n 
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where \n\ denotes the length of n. As a matter of convention, except when explicitly stated oth- 
erwise, we do not include the empty permutation in our generating functions. We are particularly 
interested in collections of permutations C which are downwards closed in the containment order, 

1. e. 7T 6 C and a < n, then a 6 C; we call such collections permutation classes. 

The two permutation classes we consider in this paper are the 321-avoiding permutations and 
the skew-merged permutations. This latter class is defined as the set of permutations which can be 
written as the union of an increasing and a decreasing subsequence. Stankova [5] proved that the 
skew-merged permutations can also be characterized as the permutations that avoid both 2143 and 
3412. This class was enumerated by Atkinson [4] via a rather intricate argument. 

Our interest is with the simple permutations in these classes, and to discuss these we need a few 
preliminary definitions. An interval in the permutation n is a set of contiguous indices I — {a, a + 
1, . . . , b} such that the set {7l(i) : i G 1} is also contiguous. Every permutation n of length n has 
trivial intervals of lengths 0, 1, and n, and other intervals are called proper. A permutation is called 
simple if it does not have any proper intervals. 

Simple permutations are precisely those that do not arise from a non-trivial inflation, in the follow- 
ing sense. Given a permutation a of length m and nonempty permutations CC\,..., a m , the inflation 
of a by ol\, . . . , cc m , denoted o~[cc\, . . . , a m ], is the permutation of length + • • • + \oc m \ obtained 
by replacing each entry c(z') by an interval that is order isomorphic to a, in such a way that the 
intervals are order isomorphic to u. For example, 

2413 [1, 132, 321, 12] = 4 798 321 56. 

It can be established that every permutation is the inflation of a unique simple permutation, called 
its simple quotient and, moreover, that the intervals in such an inflation are unique unless the simple 
quotient is 12 or 21. Permutations whose simple quotient is 12 are called sum decomposable and those 
whose simple quotient is 21 are called skew decomposable. For enumerative purposes we specify a 
unique representation of sum decomposable permutations, expressing the sum decomposable n 
as 7T = 12 [a, /3] where a is sum indecomposable. We represent skew decomposable permutations 
uniquely in an analogous way as 21 [a, /3] where a is skew indecomposable. Inflations of these two 
permutations occur frequently enough that we give them special notation, writing a ® /5 for 12 [a, /5] 
anda0^for21[a,^]. 

2. Implicit Derivation 

Before introducing our method of counting simple 321-avoiding permutations directly, we show 
how this enumeration can be obtained through an implicit relation between it and the enumeration 
of the full class. The nonempty 321-avoiding permutations have the generating function 

c(x) = 1 ~ lx ~y i Zi^ = x + 2x 2 + 5x 3 + Ux 4 + 42x 5 + 132x 6 + A29x 7 + 1430* 8 H . 

2x 

We aim to relate the generating function for simple 321-avoiding permutations of length at least 4, 
which we label s(x), to c(x) and then solve for s(x). 

The 321-avoiding permutations can be divided into four categories: 
• the permutation 1, 
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• the skew decomposable permutations, 

• the sum decomposable permutations, and 

• the inflations of simple permutations of length at least 4. 

Obviously the first category of permutations is counted by the generating function x. The skew 
decomposable 321-avoiding permutations are all skew sums of two nonempty increasing permu- 
tations, and thus are counted by x 2 /(l — x) 2 . Let /© denote the generating function for the sum 
decomposable 321-avoiding permutations. Because the 321-avoiding permutations form a sum 
closed class (re © c avoids 321 whenever n and a both do), we can decompose every sum decom- 
posable 321-avoiding permutation as the sum of a sum indecomposable permutation and another 
321-avoiding permutation, obtaining 

/© = (c-/s)c, 

which shows that 

^ 1 + c' 

Finally, every entry of a simple permutation of length at least 4 must be involved in an inversion (as 
otherwise the permutation would be sum decomposable), so to form a 321-avoiding permutation 
by inflating such a simple permutation we can only inflate by increasing subsequences. Thus we 
see that the contribution of these inflations is s(x/(l — x)). Putting this together shows that 

2 2 / 

X z C z IX 



c 2 



C(X) = X + — rrr + h S 



so we get that 



from which it follows that 



(1-x) 2 1 + c \l-xj' 
x \ (1 — 3x + 2x 2 — x 3 )c — x + x 2 — x 5 



1-x J (l-x) 2 (l + c) 



s(x) 



1 - x - 2x 2 - 2x 3 - Vl-2x- 3x 2 



2 + 2x 

The power series expansion of s(x) begins 

2x 4 + 2x 5 + 7x 6 + Ux 7 + 37x 8 + ■■■ , 
sequence A187306 in the OEIS [1]. 

3. An Iterated System for 321-Avoiding Simples 

Every 321-avoiding permutation tt has a staircase decomposition, as illustrated in the final pane of 
Figure 1. The square regions of this drawing are called its cells. For definiteness we take the ele- 
ments in the first cell to be the maximum increasing prefix T of 71, those in the second cell to be the 
maximum increasing sequence of values in n \ t, and thereafter continue according to the same 
rules. 
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Figure 1: The development of the simple permutation 24718359 6. In this development, 
hollow dots might split into groups of entries. In the first step this occurs, as the original single 
dot splits in three. That imposes two mandatory interpositions (the second and third dots in the 
second cell). There is one optional addition below it (this addition is, in fact, compulsory in the 
first step, but optional thereafter). In the next stage of the development only the last dot splits 
into two, resulting in a single mandatory interposition. One optional addition is also chosen. 



We view simple 321 -avoiding permutations as developing one cell at a time, from an initial seed 
which is a single point. At an intermediate step of this development, the elements in the final cell 
will represent either single elements of the final permutation, or groups of such elements which 
are only separated from one another by elements of the next cell. The development of a particular 
simple permutation is illustrated in Figure 1 . 

The discussion above suggests an iterated system which describes the development of 321-avoiding 
simple permutations. Specifically, suppose that we have a generating function s n (x,y) which enu- 
merates all possible developments through n cells, where the elements of the first n — 1 cells are 
weighted by x and those of the last cell by y. In the next step of the development, each "hollow 
dot" (i.e. each y) can be replaced by a sequence of one or more "filled dots" (elements). In the next 
cell, we may put a hollow dot beneath the first of these, and must put one between every pair of 
them. That is, y is replaced by 



x + xy + x 2 y + x 2 y 2 



*(y + i) 

1-xy ' 



This gives us the system 



s i(^y) = y> 

( x(y + l)\ 
s n+1 (x,y) = s„ [x,-^ , forn > 1. 

We are interested in the limit as n — » oo of this system. If we set 

y = T^ (+) 

and apply the iteration then we are at a fixed point, and so have produced the desired solution. 
This yields 

, , 1 — x — yl — 2.x — 3x 2 

six) = , 

v ' 2x 

the power series for the Motzkin numbers, 

x + x 2 + 2x 3 + Ax 4 + 9x 5 + 21x 6 + 51x 7 + 127x s + ■■■ . 
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However, the 321-avoiding simple permutations are not enumerated by the Motzkin numbers. 
What has gone wrong? 



4. Restricting to Simples 



The development procedure described above fails to generate the simple 321-avoiding permuta- 
tions correctly for two reasons. 



• The addition of an element below the first dot is optional in this system, but a simple per- 
mutation cannot begin with its minimum element, so in the first step this addition should be 
compulsory. 

• The addition of an element in the third cell to the left of the absolute minimum element (which 
is in the second cell) is allowed by this system, but this element (if added) precedes the first 
descent and thus should be in the first cell. 



It might seem that these problems would recur in later cells but they do not. In a later horizontal 
step there is no need to add an element below the smallest element of the preceding cell, since 
such elements already exist in earlier cells. Similarly, in a vertical step there is no need to forbid the 
addition of an element to the left of the minimum element, since it is already separated horizontally 
from the previous top step by at least one element of the previous bottom step. For example, the 
least element of the fifth cell is separated from the greatest element in the third cell by at least one 
element of the second cell. 

We can correct both these problems by adjusting the initial conditions. One way to do this is to 
start with a two cell system, and use a new variable z to code the least element. This gives 

S2{x,y,z 



1-xy 



We then obtain S3 by the substitutions 

x x(y+l) 

z — > and 1/ — > -. 

1 — xy 1 — xy 

This eliminates z and thereafter we use the original iteration. It follows that we should obtain the 
generating function for simple 321-avoiding permutations by substituting (t) into S3. Doing so 
yields 

, . 1 — x — Vl — 2x — 3x 2 

s \ x ) = ~ — ~ 

w 2 + 2x 

whose power series expansion begins 

x 2 + 2x 4 + 2x 5 + 7x 6 + Ux 7 + 37x 8 H , 



which (once the term corresponding to the permutation 21 is removed) agrees with the result ob- 
tained in Section 2. 
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Figure 2: The basic structure of a skew-merged permutation as described in [4]. Area I consists 
of all the elements that play the role of 1 in some 132, area II those that play the role of 1 in some 
231, area III those that play the role of 3 in some 213 and area IV those that play the role of 3 
in some 312. In general, any or all of these areas might be empty, and each is monotone of the 
type indicated by the line segments. The central region consists of all the remaining elements 
and could be either increasing or decreasing. 



5. Skew-Merged Permutations 



Recall that a skew-merged permutation is one that can be written as the union of an increasing and 
a decreasing sequence. The basic structure of such a permutation is shown in Figure 2. We intend 
to apply an analogous technique to that of the previous sections to enumerate the simple skew- 
merged permutations. In order to do so, we must refine that basic structure somewhat. The first 
thing to note is that the central area is an interval, and hence in a simple permutation can contain at 
most one point. Every simple permutation of length at least four contains either 2413 or 3142 as a 
subpermutation, so if n is a simple skew-merged permutation, then all four areas are occupied by 
at least one point. Further, it is easy to check that if n has a central element, c, and is simple, then 
n — c (the permutation obtained by n by removing c and relabeling the remaining entries) is also 
simple. We call the four elements (one from each area) closest to the centre of n its inner elements. 

To continue we assume that n has no central element and further divide into two (symmetric) 
cases depending on the pattern of the four inner elements. There are only two possible patterns 
of this type, and the following discussion assumes that we are considering a simple skew-merged 
permutation n whose inner elements have the pattern 3142 (the other possibility is its inverse, 
2413). Denote the inner elements specifically as cadb from left to right. 

Now we describe a partition of n into a sequence of monotone cells (akin to the staircase decom- 
position of 321-avoiding permutations) where each cell in the sequence interacts only with its im- 
mediate predecessor and its immediate successor. These cells are arranged in a counterclockwise 
spiral pattern, moving outwards. Their description is most easily understood by referring to Figure 
3. The first four of these cells are defined specifically as follows. 

• Ci consists of all the elements in area I lying to the right of c. In particular sgQ. 

• C2 consists of all the elements in area II which lie above some element of C\. In particular, 

b G C 2 . 

• C3 consists of all the elements in area III which lie to the left of some element of C2. In 
particular, d 6 C3. 
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Figure 3: The decomposition of a simple skew-merged permutation whose inner non-central 
elements cadb have pattern 3142 into monotone cells, each interacting with only its immediate 
predecessor and successor. 



• C4 consists of all the elements in area IV which lie below some element of C3. In particular, 
c e C4. Note also that C4 lies entirely to the left of Q by the initial choice of C\. 



Now for k > 1 the cells are defined inductively as follows. 



• Qut+i consists of all the elements in area I which lie to the right of some element of C 4 j. but do 
not belong to some previous cell. 

• Qut+2 consists of all the elements in area II which lie above some element C4,t + i but do not 
belong to some previous cell. 

• Qut+3 consists of all the elements in area III which lie lie to the left of some element of C 4 ^ +2 
but do not belong to some previous cell. 

• C4,t + 4 consists of all the elements in area IV which lie below some element of but do not 
belong to some previous cell. 



If C„ is empty for some n then the elements of n that belong to the union of the Q for k < n form 
an interval. Since n is assumed to be simple, this must be the entire permutation. 

Now we reverse the perspective above, which deconstructs a simple skew-merged permutation n 
into a sequence of cells and view it as a constructive recipe for building these permutations. As 
before we think of the cells C\ through Q_j as having been constructed for some k, along with 
some elements of Q represented by hollow dots. In extending the construction, these hollow dots 
can split into sequences which then impose mandatory insertions in the next cell, while the spaces 
between hollow dots (or before the first one in a cell) represent optional insertions. Note that the 
precise meaning of "before" depends on the area in which we are constructing the next cell, for 
example, in area II such an element would be the leftmost and greatest element of Q+i ■ We code 
the generating function for the result of applying this iteration through cell n by u n (x, y) where y 
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labels the hollow dots. Because of the necessity of including the four extreme points in the first 
four cells, the first three optional insertions at the beginning of a cell are in fact compulsory. The 
fourth one is actually forbidden (it would violate the "inner" property of c). We account for these 
variations to the general rule by adjusting the substitutions corresponding to these steps: 



Ul(x,y) 
u 2 (x,y) 

u 3 (x,y) 

u 4 (x,y) 

u 5 (x,y) 

u n+ i{x,y) 



y> 

» 2 
"3 
u 4 



X, 



x, 



x, 



l-xy 

*(i + y) 

l-xy 

*(i + y) 

l-xy 

*(i + y) 

l-xy 

*(i + y) 

l-xy 



y + 1' 

y 

y + 1' 

y 

y + 1' 
1 



y 



i' 



for n > 5. 



By computing U5 and then substituting from (t) as before, we obtain the generating function for 
the simple skew-merged permutations of this type: 



_ 1 - 2x - x 2 + (x - l)Vl -2x- 3x 2 
U ~ 2(x + l)2 • 

The generating function for all simple permutations of length at least 4 in this class is then s = 
2(x + l)u. Here the 2 accounts for the two types of inner pattern and the x + 1 for the possible 
addition of a central element. The power series expansion of s(x) begins 



2x 4 + 2x 5 



8x 6 + 16x 7 + 44x 8 



sequence A220589 in the OEIS [1]. 



Finally, we can recover the generating function, /, for the class of skew-merged permutations from 
its simple elements by the usual techniques. Skew-merged permutations are either sum or skew 
decomposable or have a simple quotient of length at least 4. Every sum decomposable permutation 
in this class is of the form 1 © n or n © 1 for a skew-merged permutation n, and the skew decom- 
posable permutation can be analogously described. Allowing for the over-counting of elements 
of the form l©7r©lorl07rQl we see that the sum or skew decomposable elements of S are 
enumerated by 

4x/-2x 2 (/+l). 

In an element with a simple quotient of length at least 4, if there is no central element in the quotient 
then the quotient can only be inflated by monotone sequences at each point. If a central element is 
present then it can be inflated by an arbitrary skew-merged permutation. Combining this with our 
previous observations gives: 



f = x + Axf - 



2x 2 if- 



l)+2u 



(/ + !)• 
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And, although it is hardly evident at a glance, this gives the known result 

i _ 3 x 

f = , = 1 + x + 2x 2 + 6x 3 + 22x A + 86x 5 + 340x 6 + 1340x 7 + 5254x 8 + ■■■ . 

(1 -2x)Vl -4x 

(Sequence A029759 in the OEIS [1].) To confirm this, we suggest that the reader do as we did and 
leave the details to a computer algebra system, but the main reason that things simplify so nicely 
is that the substitution of x/(l — x) for x in y/l — 2x — 3x 2 yields y/l — 4x/(l — x). 



6. Extensions 



It has not escaped our notice that the specific techniques we have detailed suggest possible exten- 
sions. In the language of "geometric grid classes" studied in [2, 3], the 321-avoiding permutations 
can be described as 

/ , \ 



Geom 



/ 



and we have shown that the skew-merged permutations can be expressed as 



/ 



Geom 



-l 



l -l 



-l 



V 



\ 



[J Geom 



-1 
l 



-l 



V 



It should be possible to use our techniques to enumerate the simple permutations for all infinite 
geometric grid classes which consist of a single "path", so long as this path satisfies some suitable 
regularity condition. Of course, the analysis of which hollow dots are mandatory and which are 
optional will depend on the specific grid class studied. 



References 

[1] The On-line Encyclopedia of Integer Sequences, published electronically at http: / /oeis.org/. 

[2] Albert, M. H., Atkinson, M. D., Bouvel, M., Ruskuc, N., and Vatter, V. Geometric 
grid classes of permutations. Trans. Amer. Math. Soc, to appear. 

[3] ALBERT, M. H., RuSKUC, N., AND VATTER, V. Inflations of geometric grid classes of permuta- 
tions. arXiv:1202.1833vl [math.CO]. 

[4] ATKINSON, M. D. Permutations which are the union of an increasing and a decreasing subse- 
quence. Electron. ]. Combin. 5 (1998), Research paper 6, 13 pp. 

[5] STANKOVA, Z. E. Forbidden subsequences. Discrete Math. 132, 1-3 (1994), 291-316. 



